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METHOD AND SYSTEM FOR CREATING AND ADMINISTERING INTERNET 

MARKETING PROMOTIONS 

CROSS-REFERENCE TO RELATED APPLICATIONS 

[0001] This invention relates to United States Provisional Application No. 
60/188,888, filed March 13, 2000, which is incorporated by reference herein in its 
entirety. 

FEDERALLY SPONSORED RESEARCH 
[0002] Not Applicable. 

BACKGROUND OF THE INVENTION 

Field of the Invention 

[0003] The disclosed invention relates to automatically generating marketing 
promotions for Internet websites based on real-time data obtained through controlled 
short-term experiments that determine market sensitivity. 
Description of the Background 

[0004] in traditional commerce, prices and promotions are typically static, with 
change only occurring with major market changes. This has resulted in part because 
the costs associated with printing fixed-price catalogs, marking goods with prices and 
advertising prices in the media. Furthermore, it is difficult to offer different prices to 
different purchasers in a traditional setting in which prices are published or made 
publicly available. Likewise, marketing promotions are typically somewhat static, for 
the same reasons. 

[0005] However, e-commerce does not have to be so restricted. The introduction 
of e-commerce on the Internet has made it easier for Internet merchants to change 
prices by simply updating a Web page or appropriate database/systems. The costs 
associated with printing catalogs and marking goods in a bricks-and-mortar setting are 
typically not present in eCommerce. In addition, it is also possible to offer different 
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prices to different customers without either customer learning the price that has been 
offered to the other. Likewise, it is possible to simultaneously present different 
promotional marketing campaigns to different Internet customers. 
[0006] Although it is possible for Internet merchants to update prices and other 
market variables at any time, typically they have not done so. One reason for sticking 
to static pricing strategies is that merchants are accustomed to keeping prices and 
promotions static. In some cases, merchants have both brick-and-mortar shops and 
Web shops, and want to keep prices and promotions in alignment. However, the 
primary reason why Internet merchants do not dynamically adjust market variables 
with the ever-changing marketplace is that the merchants do not have the ability to 
dynamically determine optimal values for the market variables. 
[0007] Marketing on the Internet is of a very different character than traditional 
marketing. Visiting a physical store requires an investment of time on the part of the 
customer, and there are "switching" costs associated with leaving the store, including 
the time to locate an alternative store, travel to the alternative store, and developing a 
new buyer/seller relationship. On the Internet, each of these functions is only a click 
away, and it may be very difficult to regain a customer once he leaves the website. 
Such traditional forces as geographic proximity, which draws customers to brick-and- 
mortar stores, are absent on the Internet. 

[0008] Acquiring new Internet customers is presently expensive because of the 
need to use non-Internet media, such as print or radio advertising, to inform the 
population of potential customers about the existence of the website. 
[0009] To counteract the difficulty of attracting and retaining customers, it is known 
in the art to offer promotions or incentives for customers to visit a website and 
continue making purchases. These promotions may take the form of discounts, 
coupons, prizes, product giveaways, or other free or discounted products or services. 
Lotteries, sweepstakes, contests and the like are also possible promotions. 
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[0010] Internet promotions typically differ from traditional promotions in that the 
offer made via a website to a customer is private and need not be offered to the 
general public. This is different from a traditional promotion that is offered via a 
newspaper advertisement, in which anyone possessing the ad is entitled to the 
discount. It is inefficient to offer promotions to customers who would have bought the 
product in the absence of the promotion, and therefore the Internet is a perfect place 
to customize promotions based on the identity of the customer. 
[0011] The Internet is a dynamic marketplace. As e-commerce becomes a 
dominant force, the ability to dynamically adjust to and exploit changes in the Internet 
marketplace becomes critical. An enormous amount of detailed, disaggregate 
information is being routinely captured during Internet transactions. The ability to 
gather real-time information on transactions conducted on the Internet means that 
Internet merchants could use the information to dynamically update their websites to 
take maximum advantage of market conditions. In particular, real-time transaction 
information opens up the possibility of dynamic promotions and marketing. 
[0012] However, using the information to determine dynamic, optimal market 
values is problematic. Although a great deal of real-time transactional information is 
available, businesses have no current method of being able to analyze the information 
in a manner that provides guidance to dynamically updating pricing, marketing, 
promotions and other key market variables. 

[0013] As enterprises move into high velocity environments in a networked 
economy, decisions based on data are ever more critical and can be leveraged to 
affect the bottom line. In this environment, information is highly valuable but comes 
with a high discount rate. That is, the value of the information rapidly depreciates. 
Current generation data analysis and data mining methods do not effectively deal with 
this type of information, as current methods rely on a time-consuming sequential 
process of data gathering, analysis, implementation and feedback. 
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[0014] Current systems including data mining methodologies are retrospective, and 
there is a significant lag in analysis time. The dynamic nature of the Internet makes 
even recent information obsolete. 

[0015] On the Internet, it is typical to deal with huge numbers of customers in an 
automated fashion through a webserver. It is impractical to require a human to 
evaluate the effect a promotion has on an individual customer or group of customers, 
in particular, it is not known how to the determine whether a specific customer should 
be offered a promotion or whether that customer would have bought a product at full 
price without the need for a promotion. 

[0016] It is not taught or suggested in the prior art to observe the customer's 
behavior dynamically during his period of interaction with the website to anticipate the 
need for promotion to encourage buying. It is not taught or suggested in the prior art 
to segment customers into different groups and offer different types of promotions to 
those groups depending on their observed tendency to respond to such promotions. It 
is not taught or suggested in the prior art to take real-time data to design Internet 
promotions. 

[001 7] In addition, the applicants are not aware of any prior art in which promotion 
and other market sensitivities are measured directly through use of controlled real- 
time experiments. 

[0018] In view of the foregoing, it can be appreciated that a substantial need exists 
for a method and system for dynamic optimal pricing of products and services. 

SUMMARY OF THE INVENTION 
[0019] The inability to effectively exploit Internet transaction information is 
overcome by the method and system of the present invention, which enables Internet 
businesses to conduct real-time, online experiments on a sample of transactions and 
determine marketplace sensitivities. Analysis of the results of the experiments reveal 
optimal values of key market decision variables such as price, content of banner ads, 
promotion levels, quantity discount schemes, etc. The experiments may be 
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automatically conducted on an on-going basis, or may be conducted on a periodic 
basis. The system offers total flexibility to the users to conduct and control the 
experiments. The experimental process is based upon rigorous statistical and 
econometric principles. 

[0020] An Internet merchant using the method and system of the present invention 
can control the extent and speed with which market strategies are updated. The 
method and system of the present invention preferably allow merchants to modify the 
nature of the experiment and the propagation of optimal values. Managers for the 
Internet merchant make the key business decisions, which are silently and seamlessly 
translated into the Internet merchant's eCommerce system. 
[0021] The dynamic experimentation used by the inventive system reveals the 
relative stability (or instability) of the networked market within which the business 
operates. The translation of an optimal value for a key variable (for example, 
promotion) to the entire market can be done on a real-time basis. 
[0022] Continuous real-time modeling with appropriate integration to existing 
systems on critical factors like price, promotion, financing, content, discount schemes 
and product bundling give companies using the method and system of the present 
invention a huge competitive advantage. 

[0023] In particular, the present invention allows for designing and administering 
Internet promotions via a website based on the observed characteristics and behavior 
of the customers visiting the website, 
[claim summaries] 

[0024] It is a benefit of the present invention that promotions may be designed and 

implemented without need for human intervention or attention. 

[0025] It is a further benefit of the present invention that the promotions can be 

tailored to be effective based on instantaneous data obtained from the marketplace 

itself. 
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[0026] It is a further benefit of the invention that different promotions may be 
offered to different population segments based on their observed response to 
promotion parameters. 

[0027] With these and other advantages and features of the invention that will 
become hereinafter apparent, the nature of the invention may be more clearly 
understood by reference to the following detailed description of the invention, to the 
appended claims and to the several drawings attached herein. 

BRIEF DESCRIPTION OF THE DRAWINGS 
[0028] The accompanying drawings, which are included to provide a further 
understanding of the invention and are incorporated in and constitute a part of this 
specification, illustrate embodiments of the invention and together with the description 
serve to explain the principles of the invention. 

[0029] Fig. 1 is a diagram illustrating the relationship between the sampling engine 
of the present invention and various applications that use the sampling engine; 
[0030] Fig. 2 illustrates one embodiment of a system architecture that may be used 
by the method and system of the present invention; 

[0031] Fig. 3 illustrates one embodiment of software system data flow in the 
method and system of the present invention; and 

[0032] Fig. 4 is a flowchart illustrating the process used to dynamically create and 
administer an Internet promotion strategy using the method and system of the present 
invention. 

DETAILED DESCRIPTION 
[0033] Reference will now be made in detail to the embodiments of the invention, 
examples of which are illustrated in the accompanying drawings. Wherever possible, 
the same reference numbers will be used throughout the drawings to refer to the same 
or like components. 

[0034] It is worthy to note that any reference in the specification to "one 
embodiment" or "an embodiment" means that a particular feature, structure or 
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characteristic described in connection with the embodiment is included in at least one 
embodiment of the invention. The appearances of the phrase "in one embodiment" in 
various places in the specification are not necessarily all referring to the same 
embodiment. 

[0035] The method and system of the present invention utilize limited sampling to 
determine real-time market sensitivity. This sampling provides data that can be used 
to create a real-time model that is analyzed to determine optimal values for many key 
market variables, such as price, promotions, advertising content and product 
management. The method and system of the present invention allow for customized 
dynamic promotions, discounts and bundling. 

[0036] The method and system of the present invention enables a company using 
the system to offer different promotions to different customers. The method and 
system of the present invention may be used to segment the market, and Internet 
merchants using the system of the present invention may be informed of an optimal 
promotion for each market segment. 
Architecture 

[0037] Fig. 2 illustrates one embodiment of a system architecture for the system of 
the present invention. In this embodiment, a potential customer visits a website run by 
an Internet merchant and conducts eCommerce by purchasing one or more products 
from the Internet merchant through the website. 

[0038] In the embodiment shown, the customer uses an Internet browser 201 on 
his computer to access an eCommerce site operated by an Internet merchant, such as 
Amazon.com. The Internet browser 201 may be any known to those skilled in the art, 
such as Microsoft Explorer or Netscape Communicator, for example. Preferably, 
HyperText Transfer Protocol (HTTP) or its more secure version HTTPS is used to 
communicate with the website. These are popular communication protocols used on 
the Internet to exchange information. Other communication protocols are known to 
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those skilled in the art, and are intended to come within the scope of the present 
invention. 

[0039] In an alternative embodiment not shown, the customer may be using a 
wireless handheld device to access the website. 

[0040] Once the customer has accessed the eCommerce website, he can request 
information, such as current promotions, from the website. The request sent by the 
browser might include information specific to the customer using the browser. Such 
information may include, for example, information derived from user logins, cookies 
stored on the user's machine and through the user's IP address. In addition, while the 
customer is on the website, the customer may be presented with promotions offered 
by the seller in an attempt by the seller to get the customer to purchase a product, or 
additional products. 

[0041] The customer's browser 201 communicates with an Internet merchant's 
eCommerce system 205. The eCommerce system 205 is an integrated system that 
comprises different kinds of hardware and software sub-systems. The eCommerce 
system performs the functions needed to run the Internet merchant's Website. 
[0042] Webservers are usually the entry point into an eCommerce system 205 from 
the perspective of a software program. The Webserver 210 on the eCommerce 
system is mainly responsible for delivering webpages to a browser across the Internet. 
Webpages are the pages that the user sees in the browser. The Webserver 210 runs 
software that receives and processes requests for webpages from users. The 
webpages may be stored as files on a storage disk that the Webserver reads and 
sends to the requesting browser. This is shown by 216. Alternatively, Webserver 210 
may generate the webpage by gathering information from other sources, such as 
software programs, and then send it to the browser. For example, Webpages are 
often generated with data retrieved from an Application Server. In particular, the 
Application Server may determine content for a dynamic promotion that is to be 
offered to through the webpage. 
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[0043] The Webserver 210 may be any type of known webserver, such as 
Microsoft IIS, or Netscape NES. The architecture shown in Fig. 2 also shows optional 
database 220. The database may be used by the eCommerce system 205 to store 
Internet merchant information, such as customer account records. Database 220 may 
be any known database type, such as Oracle, Sybase, DB2, etc. 
[0044] in addition, the Internet merchant may have one or more Legacy Systems 
235. For example, all customer data may be stored on a Legacy System. 
[0045] In many cases eCommerce systems interact with external systems, as 
shown by 240. For example, a trading exchange may receive catalogs from several 
external systems and store them in its own system. It may then present items from 
the catalogs to interested buyers. The eCommerce system 205 may communicate 
with external systems over the Internet or through a dedicated Frame Relay Circuit, or 
any other type of connection mechanism. 

[0046] Because Webservers usually do not perform business logic data 
processing, the architecture typically includes an Application Server 230. The 
Application Server 230 may perform most business specific logic operations and send 
data to the Webserver, which processes the data and sends formatted output to the 
user for display. For example, the Application Server may retrieve a customer's bank 
account information, which is used as part of an Order Confirmation webpage 
generated by the Webserver. 

[0047] Interprocess communications between the Application Server and the 
Webserver are typically supported by the underlying operating system. For example, 
for JAVA based platforms, the communication protocol may be RMI/IIOP (Remote 
Method Invocation/Internet Inter-ORB Protocol). The programs communicating via 
these methods may or may not reside on the same physical computer. Similar 
methods may be used for the communications between the Application Server and the 
Client Module, which is described below. 
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[0048] Communications between a Legacy System and Application Server may be 
accomplished using commercially available software, such as IBM's MQ, Microsoft's 
MSMQ or Tibco software. The software used depends on the needs and the 
underlying operating systems. 

[0049] The manager's console 265 contains software similar to browser software 
for displaying output from the inventive system to an employee of the Internet 
merchant, typically a management-level employee. It is used to manage the 
experiments run by the inventive system. It is used to configure experiments and 
display run-time progress data on the experiment. It may also be used to display data 
on past experiments. 

[0050] The client module 250 of the present invention is integrated in the 
eCommerce system 205. Client Module 250 typically consists of an Integration Layer 
251 and a Client Side Processing module 252. Collectively, it takes as input 
experiment parameter values and sends them to the Server Module 260 for 
processing. It receives output from the Server Module 260, and disseminates the 
output to the Application Server 230 and/or the Manager's Console 265 for display. 
[0051] The Client Side Processing Module 252 is responsible for processing all the 
input received from the eCommerce System, typically through the Application Server 
230, and delivering it to the Server Module 260. The input is typically a continuous 
stream of parameters used to conduct and manage an ongoing experiment. The 
Client Processing Module 252 establishes and maintains a secure communication 
channel with the Server Module and may also perform session management. 
[0052] The Integration layer 251 helps the Client Side Processing module 252 run 
on a variety of systems. It acts as an interpreter between the eCommerce System 
and the Client Side Processing module 252. It may be different for different systems. 
This enables the Client Side Processing module 252 to remain the same, no matter 
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what type of operating system is being used. In an alternative embodiment, the Client 
Side Processing module may be developed for a specific eCommerce system and 
runs without an Integration Layer. 

[0053] Communications between the Client Module 250 and the Server Module 
260 typically use HTTPS to ensure security. Data may be transmitted in extensible 
Markup Language (XML) format. 

[0054] The Dynamic Promotions System 270 includes sub-systems, computers and 
communications systems, including Server Module 260, that are used to perform the 
sampling and resultant analysis. It receives input data, performs statistical 
calculations and feeds the output to the eCommerce system 205. Typically, the 
output from the Dynamic Promotions System 270 is used by the Application Server 
230 in performing the business specific logic. 

[0055] Server Module 260 may contain Logic Module 261 , Sampling Engine 262 
and Communications Module 263. Server Module 260 is responsible for receiving 
input from Client Module 250, performing the experimentation and analysis, and 
outputting results to Client Module 250. These actions are all performed in a secure 
environment. 

[0056] Sampling Engine 262 contains statistical functionality that performs the 
various experiments. Logic Module 261 contains the algorithms that are used to 
perform various types of analyses on the sampled data. 

[0057] The Communications module 263 is responsible for securely communicating 
data to and from the Client Module. 

[0058] Database 275 may be used to store historical data and other data regarding 
the experiments for processing, report generation and future retrieval. 
[0059] The architecture shown in Fig. 2 is an ASP-based solution, where the 
Server Module 270 is hosted on a remote system with a network connection to the 
eCommerce system 205. In an alternative embodiment, the Dynamic Promotions 
System 270 may reside within the eCommerce system 205. 
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[0060] Fig. 3 illustrates how data may flow through the inventive system. 
[0061] As shown by entity 360, a management-level employee for the Internet 
merchant using the inventive system configures the Dynamic Promotions System with 
parameters. For example, the employee may enter a price range and number of 
samples to be used in the experiments. The employee may also actively monitor the 
performance of the experiment(s). 

[0062] These parameters are used as input into the Dynamic Promotions System 
as shown by data 365. These parameters configure the sampling engine subsystem 
of the Dynamic Promotions System. 

[0063] As shown by entity 301 , a customer uses a browser to access an 
eCommerce website. When the customer makes a request, several different types of 
data items may be sent to the Webserver, as shown by 305. The Webserver 
processes the information at step 310. If the request from the customer does not 
require Application Server processing, then the Webserver can go ahead and 
generate the appropriate Webpage, as shown by steps 312-315. However, if 
additional processing is needed, the Webserver will pass on information to the 
Application Server at step 320. Based on the information provided by the Webserver, 
the Application Server processes the input and performs any needed calculations at 
step 325. 

[0064] During step 325, the Application Server will determine whether it needs the 
Dynamic Promotions System to process data. For example, there may be a current 
ongoing experiment to create an optimal promotion, etc. 

[0065] If the Application Server does not need the Dynamic Promotions System to 
process information, it composes the requested information using input from its own 
calculations, databases and/or legacy systems, as shown by steps 330-335. 
[0066] Otherwise, the Application Server makes a request to the Dynamic 
Promotions System and passes on any information required by the Dynamic 
Promotions System for performing the statistical calculations, as shown by step 340. 
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[0067] The Dynamic Promotions System may use historical data in its calculations 
as shown by data 350. In addition, the parameters 365 entered by the Internet 
Merchant are used in the calculations that the Dynamic Promotions System performs. 
[0068] The Dynamic Promotions System performs the calculations as required, and 
outputs the resulting data at step 345. The Application Server composes the 
requested information at step 335 using the output from 345. 
[0069] If the manager is actively monitoring the progress of the experiment, he will 
be informed of the progress as shown by steps 370-360. 

[0070] The sampling engine 262 may be used by many different applications to 
obtain information about current market conditions. These applications use the 
sampling data to determine optimal pricing, promotions, product bundling, lead time 
discounts, quantity discounts, price versus financing and type and content of banner 
ads, for example. The Logic Module contains the algorithms to perform the different 
types of analyses required by different applications. Other applications of the 
sampling data will be known to those skilled in the art and are intended to come within 
the scope of the present invention. 

[0071] The dynamic sampling engine is the core of the inventive system. As 

shown by Fig. 5, it can be translated into modules for pricing, promotions, product 

bundling, yield management, lead time discounts, quantity discounts, price versus 

financing and banner advertisement content, for example. 

[0072] The promotions module is the focus of this application and is described 

below. 

Dynamically Creating and Administering Internet Promotions 
[0073] To maximize the effectiveness of offering a promotion, the space of 
potential promotions should be ascertained in advance. For example, sales may be 
increased significantly over a short time space by offering a deep discount (e.g. two- 
for-one sale), or equivalent coupon or gift certificate. Such promotions typically last 
only long enough to clear excess inventory or accomplish some other short-term goal. 
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Other promotions may be very long running, such as offering low interest rate 
automobile financing as a way of ensuring a steady stream of car sales. 
[0074] The present invention is particularly well suited to short-term promotions in 
which effectiveness is highly data-driven and responsive to instantaneous customer 
preferences. 

[0010] It is easy to change eCommerce prices, advertising and promotions by simply 
updating a Web page. In addition, it is possible to present different prices and 
promotions to different online customers without either customer learning the price or 
promotion that has been offered to the other. This may be accomplished by 
presenting different versions of the Webpage to different potential customers, for 
example. Because of these reasons, it is possible to perform controlled, real-time 
experiments on samples of the customer population to determine customer market 
sensitivities. This information can then be used to determine real-time optimal 
promotion and marketing strategies for an entire customer population or for selected 
segments of the customer population. In addition, merchants may learn from the 
online experiments, and apply this learning to offline counterpart market strategies. 
[0075] The sampling experiments conducted by the method and system of the 
present invention are designed to measure different customer inclinations. For 
example, one area of measurement may be to measure customer inclination to 
purchase products when offered differing promotions. In this application, promotions 
are deliberately varied by the inventive system during a sampling period, and statistics 
are kept by the system to determine what percentage of customers are likely to buy a 
product or buy additional products when offered various promotions. The statistics 
typically include, for example, the number of customers who actually purchased 
additional products at each offered promotion. 

[0076] Given the percentage of customers who buy, or who exhibit a quantifiable 
interest in buying product with each offered promotion, the system is able to compute 
an optimal or near-optimal level of promotions. The optimal promotion determined by 
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the system is intended to optimize an economic variable, such as profit. The 
economic variable to be optimized may be financial, such as profit or revenue. 
Alternatively, the economic variable may be another quantity of interest, such as 
market share, customer satisfaction, customer retention at the website, or utilization of 
manufacturing or shipping resources, for example. The optimization typically 
determines the promotion for which an economic variable is maximized, although 
other types of optimization, such as minimizing an economic variable, are possible 
using the method and system of the present invention. 

[0077] In one embodiment, the objective function may weigh multiple criterions. 
For example, the user may be trying to optimize both profit and market share. The 
objective function may be defined to be 75% weighted toward profit optimization and 
25% toward market share retention. The inventive system in this case will determine 
which promotion optimizes this weighted multi-criterion function. 
[0078] The dynamic promotion application allows companies to determine optimal 
promotions by running continuous real-time models on an appropriate sample 
population, which may be determined automatically by the sampling engine. 
[0079] Based on the objective, which may be to maximize revenue, or to reach a 
certain market share or a combination of both, once the demand function is plotted the 
system will suggest the optimal promotion. By interfacing to appropriate accounting 
systems to determine variable cost, the objective of maximizing profit (Quantity * (price 
- variable cost)) can be accomplished in real-time by the inventive system. 
[0080] Fig. 4 illustrates an overall procedure for determining an optimal dynamic 
promotion scheme. As shown in step 410, an employee for the Internet merchant first 
inputs data that is used by the inventive system to determine a sampling and 
optimization strategy. The employee is typically a manager-level employee for an 
Internet merchant. 

[0081] In this step, a parameterized space of promotions is first selected. For 
example, the amount of a discount coupon may be reduced may be a single numerical 
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parameter, as can the auto finance interest rate. In some cases, multidimensional 
promotions are possible. For example, a very large discount may be offered if the 
customer purchases immediately, a lower discount for a purchase within one day, still 
lower for purchases within on week, etc. In this example, the parameters are the 
amount of the discount and the length of time the discount will be offered. 
[0082] Other types of input may include the range of promotions that are to be 
offered, sampling intervals, and desired confidence levels. 

[0083] The employee may also input sample promotion points. For example, the 
sample promotions may include the current promotion amount, if there is a current 
promotion, and a number of specified promotion amounts both above and below the 
current promotion amount. Preferably, a sufficiently large number of promotion points 
are tested so that there are enough points to determine a smooth curve in the 
objective function. In an alternative embodiment, the employee may enter a range of 
promotion amounts, and the system may determine the promotions to be sampled. 
The system may take into account the cost of selling the products at various prices 
when determining the promotions to be offered in the experiment. 
[0084] The system may restrict the input in several ways. For example, the 
inventive system may require a minimum number of promotion points. As another 
example, the range of promotion amounts offered may be restricted to a certain 
interval by the method and system of the present invention. In this case, the sample 
sizes and desired statistical accuracies may be specified, and various mechanisms for 
limiting price changes may be implemented. For example, the promotion amount 
range may be restricted to a high promotion of selling at cost to a low promotion of 
offering no discount. The inventive system can be configured with preset price 
limitations to avoid selling or offering products at a loss greater than that desired by 
the Internet merchant, or can be programmed to require additional user confirmation 
before selling a price below a predetermined point. 
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[0085] The employee may specify whether the system is to conduct continuous 
sampling, or sample at discrete intervals. When the pricing experiment is conducted 
continuously or at varying closely spaced intervals, the prices match the instantaneous 
price customers are willing to pay. Alternatively, the experiments may be conducted 
on a regular basis, for example, the employee may set up the system to run the 
experiments on a weekly basis. 

[0086] The number of customers to sample for each promotion is also determined, 
given the promotion range and intervals. This may be an absolute number of 
customers to be presented with each promotion. Alternatively, a time interval over 
which testing is to be performed may be defined, and the customers who visit the 
website during that period become the population. A random fraction of the 
population is the sample for the experiment. In either embodiment, it may be possible 
that during the accumulation of data it will become apparent prior to the end of the 
time period or before the absolute number of customers are sampled that a particular 
promotion is optimal. In this case, since it is not necessary to continue the experiment 
to its normal completion, the sampling may be stopped. 

[0087] The Internet merchant may also determine the customer population. In one 
embodiment, the population may include every potential customer that visits the 
website. Alternatively, the customer population may be clustered or segmented, and 
only customers that meet a certain profile are sampled. As an example, customers 
may be clustered into socioeconomic groups, and only customers in certain groups 
are sampled when determining an optimal price. Alternatively, the entire customer 
population may be segmented, with separate experiments run on each segment 
determining an optimal price for each segment. As another example, customers may 
be identified for sampling based upon purchasing history or other accumulated data. 
For example, the segmentation scheme may cluster customers based on purchase 
history: heavy buyers, light buyers and non-buyers. Segments may be determined 
from a combination of demographic variables and prior purchase histories. 
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[0088] Information used to determine segmenting, or to determine which customers 
to include in the sample population, may come from outside sources, as shown by 
data storage 415. 

[0089] During the sampling process, the dynamic sampling engine 430 randomly 
samples potential customers according the parameters defined at 410. To accomplish 
this sampling, the Webserver distributes webpages 435 with the different promotions 
to the different customers in proportion to the quantities and for the period of time 
determined in step 410. These continuous dynamic experiments are used to measure 
the effectiveness of the promotions over the selected space. Effectiveness depends 
on how many customers accept the promotion, the direct cost of the promotion to the 
seller and the opportunity cost borne by the seller by giving promotions to customers 
who would have bought without a promotion, or at a lesser promotion. 
[0090] Experimentation utilizing the dynamic sampling engine 430 may be 
repeated periodically to ensure that the optimal promotion is dynamically optimized to 
regularly compensate for market changes. Thus, experiments utilizing the dynamic 
sampling engine 430 may be run monthly, weekly, daily, hourly, or more often, until 
the experimentation becomes, practically speaking, continuous. Dynamic 
optimization, therefore, is a result of continuous experimentation. The optimum 
promotion may, furthermore, be propagated to the web at 435 for offering to 
customers each time a new optimum promotion is discovered by the dynamic 
sampling engine. Alternately either the system or the operator may propagate the 
optimum promotion each time the optimum promotion changes by a particular amount 
from the previous price such as, for example, $0.25. Data from the web, such as 
purchase, timing, and use of promotions by customers may also be provided to the 
dynamic sampling engine 430 for use in future samples. 

[0091] Data is accumulated for each promotion parameter. This data typically 
include the fraction of customers who purchase the product or additional products at 
each promotion point. 
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[0092] This process is repeated until sufficient purchasing information for each 
promotion point has been obtained. If the population has been segmented, the 
sampling continues until the demand function can be estimated for each segment. 
[0093] Once the absolute number of customers, or the time period determined in 
step 410, has been sampled, the data gathered by the dynamic sampling engine 430 
is used to compute the promotion point that maximizes the economic variable(s) of 
interest. If the customers have been segmented into groups via clustering, customers 
in different clusters may have different optimal promotions, including no promotion at 
all. 

[0094] In an alternative embodiment, the promotion to be offered is not necessarily 
one of the tested promotions p 1r ... p n but is allowed to lie between two tested 
promotions. In this case, the appropriate maximization may be performed by using an 
interpolating function. 

[0095] If there are any unusual shifts in the demand function, such as a segment 
with an incorrect slope - where quantity increases when promotion decreases, or 
where quantity decreases when promotion increases - the system may automatically 
reexamine these parts of the demand function either through additional sampling or by 
taking finer intervals around these parts. 

[0096] The responses to various promotions are presented to the user at 460. In 
addition, the optimal promotion that is consistent with the objective function is 
displayed. 

[0097] In addition, the confidence intervals may be calculated. Methodologies for 
calculating confidence intervals are known to those skilled in the art. Confidence 
intervals may also be displayed to the user. 

[0098] If the experiment has not produced results that are conclusive to the 
manager, he may run the experiment again using different parameters, as shown by 
decision box 470. For example, in the experiment, the manager may have entered 
promotion points of $8, $9, $10 and $11 coupons. The experiment determines that 
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$1 1 is the optimal amount of these promotions. However, the true optimal promotion 
may be higher than $1 1 . The manager may run the experiment again using the 
promotion points of $12, $11.5, $11, $10.5, and $10 to obtain better results. 
[0099] In one embodiment, the inventive system is programmed to automatically 
update the promotion parameters if the optimal promotion calculated by the system is 
within a predefined threshold. In this case, the web server is then programmed to 
deliver promotions using the determined optimized parameters, as shown by steps 
480-485. 

[0100] The promotions that are propagated from the experiment to the customers 
may be conditioned on supplemental variables, such as length of time a potential 
customer spends visiting the site, the number of items purchased, total value of items 
purchased, prior purchasing history and seasonality. 

[0101] Consider a specific example of an optimal promotion calculated by the 
inventive system. The objective function in this example is to maximize profit. The 
Internet merchant configures the system to automatically change the price of the 
promotion if the optimal promotion determined by the system increases profit by 5% or 
more. 

[0102] In this example, the current promotion is a $6 coupon for baskets containing 
at least $50 worth of merchandise. The Internet merchant employee determines that 
the promotion amounts in the experiment should range from $5.00 to $10.00 at $1.00 
intervals. The variable cost for this promotion is estimated to be is 60% of the 
promotion amount. The minimum basket size for a promotion is set at $50. 
[01 03] A manager for the Internet merchant estimates that 1 00,000 customers visit 
the website in a day. This estimate may be made used internal data or from historical 
data, for example. The manager enters that the minimum sample size of 6%. (The 
sample size may be determined to achieve the desired levels of confidence intervals.) 
Therefore, of the 1 00,000 customers that visit the website, the dynamic sampling 
engine will pick out 6,000 to receive the webpages generated with the different 
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promotions used in the experiment. The other customers who are not selected for the 
experiment receive the standard promotion, if any. In this example, 1 ,000 random 
customers will receive webpages for each of the 6 promotion points. 
[0104] Other methods of sampling are known to those skilled in the art, and are 
intended to come within the scope of the present invention. 

[0105] Table 1 illustrates a demand table calculated by the system of the present 
invention for this example. 



Table 1 



Promotion 
Amount 


Estimate 
Variable Cost 


Number of promotions 
accepted 


Basket Size 
(mean value) 


Incremental 
Profit 


$5.00 


$3.00 


144 


$77 


$1,900.80 


$6.00 


$3.60 


156 


$81 


$2,340.00 


$7.00 


$4.20 


187 


$80 


$2,580.60 


$8.00 


$4.80 


290 


$83 


$4,350.00 


$9.00 


$5.40 


295 


$85 


$4,602.00 


$10.00 


$6.00 


302 


$83 


$4,167.60 



[0106] Incremental profit is calculated by the equation: 
profit = (Number of offers promotions accepted) * 

(basket size - promotion amount - minimum basket size) * 
(1 - variable cost) 

[0107] For example, the incremental cost of offering $5 promotions in this example 
can be calculated by: 

(144)(77 - 5 - 50)(1 - .4) = $1 ,900.80 
[0108] Table 1 shows the estimated incremental profit for each of the promotional 
amounts offered. As shown in Table 1 , the promotional amount of $9 will maximize 
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the profit in this example. This is a 15.5% increase over the current incremental profit 
of 2,340.00, so the user's Website is automatically updated to use the optimal 
promotional amount of $9. 

[0109] In a further embodiment of the current invention, the value of the promotion 
varies based on the perceived need to retain the customer at the website. 
[01 10] In still a further embodiment, the value of the promotion depends on the 
value of the items currently in the customer's market basket. That is, the customer 
may be offered promotions of ever-increasing value to induce further buying behavior. 
[011 1] It should be apparent that the present invention is not confined to 
promotions, but in fact may be used to design and administer any type of alteration in 
the offer made by a seller to a buyer over the Internet, including but not limited to, 
promotions, warranty terms, quality levels, delivery terms, quantities or any other 
contractual term. All that is required is that the response of the customers to the offer 
be experimentally testable over the Internet and the altered terms be capable of 
communication over the Internet. 

[0112] The above example illustrates one method for sampling the population, 
other methods are known to those skilled in the art. For example, the size of the 
random sample may be determined based on the manager's levels of confidence 
intervals. Many statistical methods are known to those skilled in the art. Furthermore, 
the following statistical references are incorporated by reference in their entirety: (a) 
Ross (1997), A First Course in Probability, Prentice Hall, Upper Saddle River, NJ; (b) 
Gelman A., J. B. Carlin, H.S. Stern and D.B. Rubin (1995), Bayesian Data Analysis, 
Chapman & Hall, New York, NY; (c) Malhotra, N.K. (1993), Marketing Research, 
Prentice Hall, Englewood Cliffs, NJ; (d) Wedel, M and W.A. Kamakura (1998), Market 
Segmentation: Conceptual and Methodological Foundations, Kluwer Academic 
Publishers, Boston, MA; (e) Pudney (1989), Modelling Individual Choice: The 
Econometrics of Corners, Kinks and Holes , Basil Blackwell Limited, Oxford, United 
Kingdom; (f) Cinclair E. (1975), Introduction to Stochastic Processes, Prentice-Hall, 
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Englewood Cliffs, NJ; (g) Kalbfleisch, J.D. and R.L. Prentice, The Statistical Analysis 
of Failure Time Data, John Wiley & Sons, New York, NY; and (h) Mitchell, T.M (1997), 
Machine Learning, McGraw- Hill, New York, NY. Those references describe statistical 
methods that may be utilized by the present invention. 

[0113] While the invention has been described in detail and with reference to 
specific embodiments thereof, it will be apparent to one skilled in the art that various 
changes and modifications can be made therein without departing from the spirit and 
scope thereof. Thus, it is intended that the present invention cover the modifications 
and variations of this invention provided they come within the scope of the appended 
claims and their equivalents. 
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